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ABSTRACT 

the evaluation of Project Symbiosis which focused on enhancing the 
teaching of science principles in high school agriculture courses. 

The project initially involved 15 teams of science and agriculture 
teachers and was characterized by an extensive evaluation component 
consisting of six formal elements and one informal element. Data 
collected for evaluative purposes included preparticipation 
information, workshop evaluations, post-workshop telephone 
interviews, teacher generated products, and on-site interviews. 
Preliminary implications of Project Symbiosis suggest that evaluation 
methods must be multifaceted, the interdisciplinary teacher teaming 
requires extended periods of time, and structures within schools 
should be altered to assist teams. (DDR) 
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Evaluation of Project Symbiosis: 

An Interdisciplinary Science Education Project 

Based on a presentation by James W. Altschuldfor the panel entitled "'The Changing Role of Evaluation in 
Reforming Science Education " at the Annual Meeting of the American Evaluation Association 



Overview of the Project and 
Evaluation Considerations 

In 1990, three faculty members | 

(Rosemarie Rossetti, Projea Direaor; ' 

N.L. McCaslin; and Wesley Budke) in | 
the Department of Agricultural Educa- j 
tion at The Ohio State University and 
James Altschuld, Evaluation Coordina- 
tor at NCSTL began to design a projea to j 
form interdisciplinary teams of second- ! 
ary science and agriculture teachers. 

[Note: It must be emphasized that the 
projea was a team effort of the four 
individuals identified above and two 
Graduate Research Associates, Carol 
Bezek and Inyoung Kim, of the NCSTL 
who construaed and analyzed the data 
base.l The main purpose was to 1 

enhance the teaching of science t 

principles in high school agriculture j 
courses. Agricultural education was 
chosen due to the faa that knowledge 
and application of varied science 
principles are required for successful 
farming as well as other agriculturally 
related business and industry endeavors. 

It was also felt that the teaching of ; 

science in agriculture needed to be i 
improved in light of the necessity to | 

teach many diverse issues and topics in ' 
agriculture. 

The key features of the projea were 
as follows: ] 

a) recruitment via established I 

agricultural education networks of 
agricultural teachers from throughout 
the State of Ohio to participate with 
their personally identified science 
teaching counterpart; 

b) the development of five, day-long 
workshops to be implemented on a 
monthly basis starting in the Fall of 1991 | 
with the last one taking place in 
February of 1992; 

c) the requirement that teachers 
make a commitment to take part in a 
large set of evaluation procedures 
designed for the projea and that they 
develop and, if possible, try out assign- ! 
ments, demonstrations, team teaching, ; 
or other results of their interdisciplinary i 
teaming. 

In line with these features 1 5 teams 
were eleaed or chosen to be involved 
during the summer of 1991. The faculty ; 
from Agricultural Education and the I 



Evaluation Coordinator planned the five 
workshop programs recognizing that 
various science disciplines and different 
interests in agriculture were represented 
in the sample of participants. The 
presenters were from agribusiness and 
industry, the university, government, 
school systems, and science organiza- 
tions. Topics included water quality, the 
properties of the egg, advances in plant 
science, animal genetics and breeding, 
biotechnology, environmental manage- 
ment, ideas for team work, and the 
evaluation of the projea. 

As was expeaed most of the teachers 
were from smaller, more rural schools. 
The projea paid for: substitutes while 
the teachers attended the workshops: 
transportation to and from workshops; 
and per diem expenses. In addition, a 
nominal amount of support was 
provided for teacher aaivities in schools. 

Originally, a somewhat extensive 
evaluation was proposed for the projea. 
Upon review of the proposal the sponsor 
requested that the evaluation be 
expanded and accordingly the evalua- 
tion struaure contained six formal 
elements and one informal one. The 
formal elements were: (1) colleaion of 
pre-participation information regarding 
background variables, expeaations and 
perceptions of the concept of interdisci- 
plinary teaming; (2) evaluations of each 
workshop via closed and open-ended 
questions; (3) telephone interviews with 
half of the team members after the first 
workshop and similar interviews with 
the other half after the second work- 
shop; (4) teacher maintained monthly 
log sheets of aaivities; (5) colleaion and 
analysis of teacher generated produas; 
and (6) on-site interviews conduaed 
with six of the teams in April and May. 
The informal element consisted of 
evaluator and faculty observations made 
during the course of events such as 
noting the comments of teachers. The 
evaluation philosophy expressed to 
teachers at the initial workshop was that 
the projea was developmental in nature 
and their feedback and ideas were 
welcomed. They were encouraged to be 
open and frank in their comments. 

Given the scope of the evaluation, 
results from a preliminary analysis of the 



data were submitted to the Kellogg 
Foundation in 1992. Since that time we 
have construaed a data base system that 
facilitates in-depth probing related to 
questions about the nature of interdisci- 
plinary teaming. In this manner, 
insights should be gained for the 
development of subsequent projeas of 
this type based upon better understand- 
ing of the interdisciplinary teaming 
process. 

Now, let us focus on the multifaceted 
considerations that go into designing an 
evaluation of a teamwork projea such 
as this one. A description of seleaed 
major results will also be given. 

Evaluation Design 
Considerations 

First, the question arises as to what is 
meant by the concept of interdisciplinary 
teaming and what should be the end 
result or outcome? What is the nature 
of the criterion in this projea both in 
terms of process and produa? What are 
likely expeaations given the less than 
one year time frame? What would 
demonstrate that the projea has had an 
impaa and what type of impaa should 
it be? Projea staff frequently discussed 
outcomes including changes in teacher 
behavior, concrete evidence that the 
interdisciplinary work has affeaed 
course content and instruaion, en- 
hanced teacher perceptions and under- 
standings regarding the substantive 
content of the workshops, conceptual 
changes in how teachers view teaching, 
effeas on others within the school 
setting, effeas on student learning, clear 
evidence of an interdisciplinary out- 
come, and sustainability of the team 
beyond the conclusion of the projea. 
Many possibilities exist with each one 
having unique implications for evalua- 
tion design and implementation. 

The questions just examined reflea 
different perspeaives of the goals of the 
projea. The position adopted in this 
evaluation is that the goal was to 
enhance interdisciplinary teaming. In 
other words, the teacher is the main 
recipient of projea services and that's 
I where the emphasis of the evaluation 
I should be. The other position would be 
j that the students are the ultimate 
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beneficiaries and the evaluation should 
ascertain their views, feelings and 
knowledge changes. 

Second, another concern is to what 
extent is it possible to describe the 
process of interdisciplinary teaming in 
the schools. (Description of the process 
was important for purposes of generali- 
zation.) The 15 teams were spread 
throughout the state and limited 
evaluation dollars and time were 
available to support on-site visits. Also it 
is unclear as to what should be observed 
during site visits especially if teachers 
were in an early discussion, idea 
generating, planning mode. Under such 
conditions and with the recognition that 
monitoring of the process was desirable, 
proxy measures were used in place of 
direct observation and investigation. 
Phone interviews, teacher logs, and 
informal discussions and observations at 
the workshops were the main sources of 
data. 

Third, how much data can realisti- 
cally be colleaed from busy full time 
teachers without becoming a resented 
imposition. Over time, it is likely that 
the completion of forms and logs would 
become perfunaory. What is the 
quality of logs if they were produced 
'after-the-faa' and represented retro- 
speaive recall more than data immedi- 
ately written at the occurrence of an 
event? Thus the reliability and validity 
of the information colleaed in general, 
and teacher logs, in particular, are 
vulnerable to question in this regard. 

Fourth, it may be difficult to 
combine information from multiple 
sources of data into a meaningful, 
coherent picture. The data generated 
comes from a teacher charaaeristics 
form (n=30 teachers from 15 teams), pre 
workshop questionnaires (n=30) which 
established a baseline of teacher 
perceptions and expeaations, five fairly 
in-depth workshop questionnaires with 
an average completion rate of more 
than 85 percent, 30 telephone inter- 
views completed over a two month 
period, more than seventy five log 
sheets submitted by teams or individual 
team members over four separate time 
periods, teacher produced produas and 
plans, and six on-site interviews with 
teacher teams that lasted about one and 
one half hours. This rich base contains 
extensive qualitative and quantitative 
information. 

Selected Findings from 
Project Symbiosis 

• One interesting and somewhat 
surprising finding was that there was 



limited prior team work experience and 
especially in regard to serious attempts 
at interdisciplinary teaming. The 
literature also contained few examples 
of or studies related to interdisciplinary 
teaming at the secondary level. 

• Another finding was that most 
teachers who participated were highly 
experienced (more than 12 years on the 
average) possibly indicating that one had 
to feel fairly comfortable in the teaching 
role before embarking on a teaming 
approach. Another related interpreta- 
tion might be that with experienced 
teachers it may be more difficult to 
change or alter established teaching 
routines. 

• Not unexpeaedly teachers consis- 
tently showed the highest preferences 
for workshop presentations that 
explained the basic theoretical, concep- 
tual underpinnings for a principle 
followed by hands-on opportunities that 
could be applied to their classrooms. 
From an analysis of written comments 
and scaled items, teachers were satisfied 
with the content chosen for the work- 
shops and the quality of presenters. 

• Notably lower preferences were 
expressed for those parts of the work- 
shops that dealt with teaming strategies 
and issues. This may reflea the quality 
of the presentation, underlying teacher 
needs and expeaations, and/or a 
combination of both of these faaors. 

• A portion of the teams did achieve 
a measure of interdisciplinary aaivity 
whereas others demonstrated limited 
achievement in this regard. Several 
teams dropped out citing not the basic 
premises of the projea but severe time 
constraints affeaing their ability to 
attend workshops, to meet as a team 
and to complete assignments. 
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• Systemic problems were noted 
throughout the evaluation process. 

Many of the teachers did not have 
common free periods in which to discuss 
teaming aaivities and large blocks of 
planning/ meeting time were not in 
evidence. Teachers did not share the 
same students nor, as a general rule, 
were they located in physical proximity 
to each other. Meetings were generally 
of short (under 20 minutes) duration. 
Most teachers were familiar with but 
had not observed their teammate 
teaching a class. Schools do not have 
formal provisions for observing and 
exchanging ideas about teaching. The 
agriculture teacher, as anticipated, 
almost always had more flexible space 
and physical arrangements than did the 
science teacher. While administrative 
facilitation was not present to a major 
degree, administrative hindrance also 
was not there. 

Preliminary Implications of 
Project Symbiosis 

Evaluation must be multifaceted. 

Single instruments or approaches to 
evaluation will not provide the quality 
and depth of information necessary for 
understanding what is taking place. 

Interdisciplinary teaming re- 
quires an extended period of time. 

If we learned one thing, it is that this 
teaming process requires a great deal of 
time for teachers to get acquainted, to 
conceptualize, to develop, and to 
implement ideas. Allot two years or 
more. 

Interdisciplinary teams need 
facilitation. Teams don't occur by 
themselves. A projea base or central 
facilitation mechanism has to be there. 

Administrators need to be 
involved in interdisciplinary 
teaming. The involvement should 
relate to appreciating the process and its 
value, developing teacher schedules to 
accomodate teaming, and reinforcing 
the process in any way possible. 

Structures within schools should 
be altered to assist teams. Planning a 
year in advance is desirable for establish- 
ing time (free periods) for teachers to 
work together, for scheduling groups of 
students common to teachers, as well as 
for making other provisions to enhance 
the team performance. 



Initial funding of this effort by the 
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